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2011  Assessment  Made  Observations 
On  The  Following 


•  Cost  characteristics  of  the  MDAP  portfolio 

•  Timing  and  amount  of  knowledge  achieved 

•  Progress  of  WSARA  implementation 

•  Progress  of  DOD  efficiency  initiatives 
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Cost  Characteristics  of  DOD’s  Portfolio 
of  Major  Defense  Acquisition  Programs 
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08  to  10:  Portfolio  $  Investment  Same, 
Programs  Grew  By  a  Net  of  Two 


*  15  programs  estimated  at  $77  billion  entered 

*  13  programs  estimated  at  $174  billion  exited 


Portfolio  status 

Fiscal  year 

Fiscal  year] 

2008 

2010 

Number  of  programs 

96 

98 

Total  p.anned  investment 

$1.6 . 

$1.68  trillion 

Funding  expended 

$834  billion 

$968  billion 

Fundin  to  com  lete 

$802  billion 

$712  billion 

Source:  GAO  analysis  of  December  2007  and  December  2009  Selected  Acquisition  Reports. 
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2yr/5yr/Baseline  Trend:  FY  2010  MDAP 
Portfolio  Cost  Growth  Over  Time 


FY  2011  dollars 

Last  2  years 

Last  5  years 

Since  first  full 
estimate 

(2008  to  2010) 

(2005  to  2010) 

(Baseline  to  2010) 

Increase  in  estimated 

$15  billion 

$29  billion 

$102  billion 

RDT&E  costs 

5  percent 

10  percent 

47  percent 

Increase  in  estimated 

$121  billion 

$186  billion 

$287  billion 

procurement  costs 

11  percent 

18  percent 

31  percent 

Increase  in  total 

$135  billion 

$217  billion 

$402  billion 

acquisition  cost 

y  percent 

16  percent 

35  percent 

Average  delay  in  delivering 

5  months 

9  months 

22  months 

initial  capabilities 

8  percent 

13  percent 

30  percent 

Source:  GAO  analysis  of  December  2009  Selected  Acquisition  Reports. 
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RDT&E  Percentage  Cost  Growth  From 
Baseline  per  MDAP 


325% 

275% 

225% 

175% 

125% 

75% 

25% 

-25% 
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Summary  Analysis 

Average  weighted  growth  =  47  percent 
Median  growth  =  21  percent 


.wic.  Cost  Growth  =  $102  billion 


in 
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Note:  Four  programs  have  greater  than  325  percent  RDT%E  cost  growth.  The  fourg 
programs  that  exceed  325%  range  from  348%  to  3633%. 
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Impact  of  Quantity  INCREASES  on 
Program  and  Portfolio  Cost 

43  MDAPS  had  increased  quantities  since  starting 

•  Total  quantities  for  all  increased  by  73% 

•  Overall  total  program  cost  increased  by  100% 

•  A  “calculated”  cost  for  increased  quantities  is  $175B 

•  The  actual  cost  increase  was  $258B 

•  The  difference--$83B--can  be  thought  of  as 
inefficient  cost  growth  or  “bad”  cost  growth 
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Impact  of  Quantity  DECREASES  on 
Program  and  Portfolio  Cost 

•  30  MDAPS  had  decreased  quantities  since  starting 

•  Total  proc  cost  actually  increased  fui  10  pi oyi arris 

•  A  “calculated  savings”  for  decreased  Q  is  $197B 

•  The  actual  cost  INCREASED  by  $2B 

•  The  difference--$199B — can  be  thought  of  as 
lost  buying  power 
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Timing  and  Amount  of  Technology, 
Design,  and  ManuTaciunng  rvnowieage 

Achieved 


A  GAO 
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A  Knowledge-Based  Approach  is  Key  to 
Successful  Program  Outcomes 


Materiel  Development 
Decision 

o 


Knowledge- 

Based 

Model 


Technology 

development 


5  to  6  years  or  less 


Development  Start 


Production  Start 
/\ 


o 

PDR 


Integration 


t 


Product  development 

Demonstration 


o 

CDR 


Production 


t 


Knowledge  Point  1 
Technologies,  time,  funding  and 
other  resources  match  customer  needs. 


Knowledge  Point  2 
Design  is  stable  and  performs 
as  expected. 


Knowledge  Point  3 
Production  meets  cost,  schedule, 
and  quality  targets. 


Decision  to  invest  in  product  development. 


Decision  to  start  building  and  testing  Decision  to  produce  first  units  for 

production  representative  prototypes.  customer. 


•  Model  provides  framework  for  incremental,  time  certain  (development  constrained  to  5  to  6  years 
or  less),  and  knowledge-based  approach  to  weapon  system  acquisitions. 

•  Success  requires  structured,  disciplined  application  and  adherence  to  model. 

•  Knowledge  points  align  with  key  investment  inflection  points. 

•  Controls  are  in  place  for  decisions  makers  to  measure  progress  against  specific  criteria  and 
ensure  managers  capture  key  knowledge  before  moving  to  next  phase. 
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Focus  on  Several  Knowledge-Based 
Practices  at  Development  Start 


Knowledge-based  practices  at  development  start 

IAMD 

Knowledge  point  1 

Mature  all  critical  technologies 

O 

Hold  system  requirements  review 

• 

Hold  system  functional  review 

• 

Hold  preliminary  design  review 

oa 

Constrain  development  phase  to  6  years  or  less 

o 

^  Practice  implemented  by  program 
O  Practice  not  implemented  by  program 
Source:  GAO  analysis  of  DOD  data. 
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Design  Knowledge  Increasing,  but 
Prototypes  Are  Not  Being  Used 


Knowledge-based  practices 
at  design  review 

Knowledge  point  2 


Mature  all  critical  technologies 

o 

o 

o 

• 

o 

o 

o 

• 

• 

Release  at  least  90  percent  of  design  drawings 

0 

o 

• 

• 

o 

• 

• 

• 

• 

Test  a  system -level  integrated  prototype 

o 

o 

o 

o 

o 

o 

o 

o 

Use  a  reiiability  growth  curve 

o 

o 

o 

o 

• 

• 

o 

• 

o 

Conduct  producibility  assessments  to  identify 
manufacturing  risks  for  key  technologies 

• 

• 

• 

• 

• 

• 

Complete  failure  modes  and  effects  analysis 

• 

• 

• 

• 

• 

• 

• 

^  Practice  implemented  by  program 

O  Practice  not  implemented  by  program 

Practice  not  applicable  or  information  not  available 
Source:  GAO  analysis  of  DOD  data. 
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Programs  Are  Identifying  Processes, 
But  Not  Demonstrating  Them  Pre-Prod 


Knowledge-based  practices 
at  production  decision 

Knowledge  point  3 


o  /  ,  /SO 

Jf  /  #  / 

O  /  <o  /  k/ 


Mature  all  critical  technologies 

• 

• 

• 

o 

• 

o 

• 

• 

• 

• 

Release  at  least  90  percent  of  design  drawings 

• 

• 

• 

• 

o 

• 

o 

Identify  key  product  characteristics 

• 

• 

o 

• 

• 

• 

• 

• 

• 

Identify  critical  manufacturing  processes 

Demonstrate  critical  processes  are  in  statistical 
control 

o 

o 

o 

o 

o 

o 

o 

o 

o 

Demonstrate  critical  processes  on  a  pilot 
production  line 

• 

• 

• 

• 

o 

• 

• 

• 

• 

Test  a  production-representative  prototype 

• 

• 

• 

o 

o 

o 

• 

o 

o 

• 

^  Practice  implemented  by  program 

O  Practice  not  implemented  by  program 

Practice  not  applicable  or  information  not  available 
Source:  GAO  analysis  of  DOD  data. 
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Progress  of  Acquisition 
Reforms  and  Efficiency  Initiatives 
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New  DOD  Policies  Could  Improve 
Outcomes 

*  More  discipline  and  up-front  knowledge  in  early  acquisition 
phases  could  put  programs  on  more  stable  footing 

•  Early  Materiel  Development  Decision  required  for  all  programs. 

•  Preference  for  incremental  development,  with  baselines  for 
each  increment. 

•  PDR  required  before  system  development  start. 

•  competitive  prototyping  required  as  part  ot  tecnnoiogy 
development  phase. 

•  Configuration  Steering  Boards  established  to  control 
re  uirements  cree.  . 

•  Acquisition  strategies  required  to  describe  measures  taken  to 
ensure  competition  throughout  the  program  lifecycle. 

•  Trade-offs  among  cost,  schedule,  and  performance  objectives 
required  at  Milestone  B  approval  to  ensure  affordability. 
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Programs  Have  Begun  to  Implement 
DOD’s  Revised  Acquisition  Policies 

•  Programs  in  our  2011  assessment  have  begun  to  implement 
acquisition  reforms  that  could  improve  cost  and  schedule  outcomes. 

•  Competitive  prototyping  -  9  of  14  pre-MDAPs  planned  to  develop 
competitive  prototypes  prior  to  Milestone  B. 

•  Early  systems  engineering  -  10  pre-MDAPs  in  our  assessment  have 
already  scheduled  a  preliminary  design  review  before  Milestone  B. 

•  Trade-offs  -  7  of  14  programs  reported  making  major  cost,  schedule, 
and  performance  tradeoffs  before  development  start 

•  Competition  -  6  of  14  programs  are  planning  to  incorporate 
competition  into  their  acquisition  strategy  after  Milestone  B 

•  Several  programs  in  our  2011  assessment  still  have  not  reported 
holding  a  configuration  steering  board  meeting. 

•  12  of  40  programs  in  our  assessment  reported  never  having  held  a 
configuration  steering  board. 

•  5  programs  presented  de-scoping  options  to  the  board  and  4  had 
those  approved  to  help  maintain  cost  and  schedule. 


16 


GAO 

Accountability *  *  Integrity  *  Reliability 


DOD  Efficiency  Initiative  Can  Help 
Further  Reforms 

•  Sets  shorter  programs  timelines  -  Requirements  and  proposed 
schedules  must  be  consistent;  justification  for  proposed  program 
schedule  is  required  before  a  program  can  proceed. 

•  Treats  affordability  as  a  requirement  -  Affordability  is  to  be 
treated  like  a  key  performance  parameter  at  Milestone  A. 

•  Stresses  the  use  of  systems  engineering  analysis  -  At 

Milestone  B,  re  ,uires  the  ,  resentation  of  a  systems  engineering 
tradeoff  analysis  showing  how  cost  varies  with  schedule  and  design 
parameters. 

•  Emphasizes  competition  throughout  the  program  lifecycle  - 

Requires  the  presentation  of  a  competitive  strategy  at  each 
program  milestone 

•  Recommends  portfolio  analyses  to  eliminate  redundancies  - 

Conduct  portfolio  reviews  at  the  joint  and  Department-wide  level  to 
identify  redundancies,  as  well  as  among  smaller  programs. 
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